The Effect of the Statistical Characteristics of Input Pattern Vectors on Neural Learning: an Experimantal Study

To achieve compactness in the final weights is to be of high interest in terms of generalization ability of neural networks. In literature, techniques such as weight elimination, weight decay have been proposed and a tendency to decay to zero for the connections between neurons is subsequently employed in order to obtain a smaller network which improves the generalization capability. Within this paper a new method to achieve compactness in the final weights is proposed. The idea is based on discovering the link between the statistical characteristics of the input pattern vectors and that of the final weights.